Indonesian Journal of Electrical Engineering and Computer Science 
Vol. 16, No. 1, October 2019, pp. 539~543 
ISSN: 2502-4752, DOI: 10.1159 1/ijeecs.v16.i1.pp539-543 0 539 


Hybrid conjugate gradient parameter for solving symmetric 


systems of nonlinear equations 


M. K. Dauda', Mustafa Mamat”, Mohamad A. Mohamed’, Nor Shamsidah Amir Hamzah* 
'2.3Raculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Malaysia 
‘Department of Mathematical Sciences, Kaduna State University, Nigeria 
“Faculty of Applied Science and Technology, Universiti Tun Hussien Onn, Malaysia 








Article Info 


ABSTRACT 





Article history: 


Received Nov 1, 2018 
Revised Feb 6, 2019 
Accepted Mar 15, 2019 





Keywords: 


Conjugate Gradient 
Derivative Free 
Hybrid 

Nonlinear Equations 
Symmetric System 


Mathematical models from recent research are mostly nonlinear equations in 
nature. Numerical solutions to such systems are widely needed and applied in 
those areas of mathematics. Although, in recent years, this field received 
serious attentions and new approach were discovered, but yet the efficiency of 
the previous versions suffers setback. This article gives a new hybrid conjugate 
gradient parameter, the method is derivative-free and analyzed with an 
effective inexact line search in a given conditions. Theoretical proofs show 
that the proposed method retains the sufficient descent and global convergence 
properties of the original CG methods. The proposed method is tested on a set 
of test functions, then compared to the two previous classical CG-parameter 
that resulted the given method, and its performance is given based on number 
of iterations and CPU time. The numerical results show that the new proposed 
method is efficient and effective amongst all the methods tested. The graphical 
representation of the result justify our findings. The computational result 


indicates that the new hybrid conjugate gradient parameter is suitable and 
capable for solving symmetric systems of nonlinear equations. 
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1. INTRODUCTION 

In recent years, many researchers [1-5] focused on the techniques which resulted to emergence of 
several CG formulas coefficient (8 _k) in solving unconstrained optimization problems, but hybridizing the CG 
coefficient (B_k) gives so many advantages. Hybrid CG parameters are basically designed based on an adaptive 
switch from a CG parameter, these includes the hybridizations of PRP and FR methods in proposed by Touati- 
Ahmed and Storey [6], Hu and Storey [7] and Gilbert and Nocedal [1], the hybridizations of HS and DY 
methods proposed by Dai and Yuan [8] and the hybridization of LS and CD methods proposed by Touati- 
Ahmed and Storey [6]. In the hybrid CG methods of [6, 8], the CG parameter is computed based on discrete 
combinations of the CG parameters of the two categories. Recently, Andrei [9-13] proposed several efficient 
hybrid CG methods based on convex combinations of the CG parameters of the two categories which are 
continuous combinations. More exactly, in [9] a hybridization of HS and DY methods has been proposed in 
which the hybridization parameter is computed based on the standard secant (quasi-Newton) equation [14, 15]. 
In [10], using an acceleration scheme, Andrei proposed another hybridization of HS and DY methods in which 
the hybridization parameter is computed based on a modified secant equation proposed by Li et al. [3]. The 
hybridization parameter in our methods is computed from a modified secant equation obtained based on the 
search direction of the Hager-Zhang nonlinear CG method. The survey by Hager and Zhang (2006) discussed 
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extensively on some methods with special emphasis on their global convergence. The main aim of this 
proposed hybrid method is to find out new formula for conjugate methods such that they are not only globally 
convergent for general functions but also have good numerical performance. 


2. DERIVATION OF THE HYBRID CONJUGATE GRADIENT PARAMETER FOR SOLVING 
SYMMETRIC SYSTEMS OF NONLINEAR EQUATIONS 

In this section, the new formula was derived from [16] and [17], the resulting hybridizations formula 
is given below. The attractive features of these methods are to attain good computational performance and to 
maintain the attractive feature of strong global convergence. In order to take advantage of these, a 
hybridizations of these methods is carried out using the Andrei’s approach together with the nonnegative 
restriction of the CG parameters suggested by Powell [5]. 

From [16], the CG parameter is given by 
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Also, from [17], the CG parameter is given by 
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Considering the concept of [18,18,20], the new CG parameter f, is given by: 


By = 1 — @) + @B,,, where a is the hybridization parameter and is a scalar satisfying 0 < a < 1. If 
a= 1, then Br = Bre while Br = By, if a = 0. 


Thus, 





OnVR-Sk). ~s,)? T 
B= - a) § kVk St) Ik+1 4 oy Ok 3k Cras ae 1,2,3,:, 0, = Sek GB) 
Ve Ok Vi Ak SKVk 


The direction d;, is assumed to be a decent one and is obtained using 
—F(x,) ifk =0 

a, ={ | 4 
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where f; is term as conjugate gradient parameter. The CG methods for solving nonlinear systems of equations 
generates an iterative points {x,} from initial given point Xo via 


Xes1 = XK + Ved (5) 


where y, > 0 is attained via line search. Using the above procedure, the following is the algorithm for the new 
update. 


Algorithm: 

Step 1: Given x9, vy > 0,a = 0.6, ce(0,1) ande > 0 compute dg = —F(%Q), set k = 0. 

Step 2: Compute F (x,) and test the stopping criterion, i.e. ||F(x;)|| < €,if yes, then stop, otherwise 
continue with next step. 

Step 3: Compute y;, by using the line search in [16,17,21,22] 

T 
tee and new Bx. 
Step 5: Compute X,4, = x, + ad, 

Step 6: Compute search direction using (4) 
Step 7: Set k = k + 1 and go to step 2. 





Step 4: Compute 6, = 
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3. NUMERICAL RESULT 

In this section, the numerical results of the proposed method is given. The proposed hybrid method is 
denoted as M1, and its performance is compared with the classical conjugate gradient parameter methods 
denoted as M2 and M3 respectively. In Tables | and 2, each n-dimensional problem with initial starting point 
is considered as one problem, therefore a total of 40 benchmark problems were solved. The computational 
experiment is based on the number of iterations and CPU time. To ascertain the global convergence of the 
proposed method, the benchmarks problems in [21, 22] was used with two different ISP, and the output of the 
performance of the methods was based on the performance profile presented by Dolan and More [24]. 


Table 1. Performance profile of M1, M2 and M3 based on iteration number & CPU time 








Prob ISP Dim Mil M2 M3 
Tter CPU Iter CPU Tter CPU 

la (0.5,0.5, ++,0.5)"7 10 1 0.006806 13 0.015034 17 0.014147 
100 1 0.000632 14 0.002887 18 0.002249 

500 1 0.000366 14 0.003004 18 0.008648 

1000 1 0.000433 14 0.003024 19 0.009221 

5000 1 0.001403 14 0.123704 21 0.057777 

1b (0.2,0.2, +++ ,0.2)" 10 5 0.000567 10 0.001078 15 0.001418 
100 6 0.001142 10 0.001165 18 0.003607 

500 6 0.001000 11 0.001951 18 0.004433 

1000 6 0.001395 Il 0.004255 18 0.007740 

5000 7 0.006488 11 0.013589 21 0.055354 

2a (0.5,0.5, +++,0.5)"7 10 6 0.000580 4 0.000809 18 0.001765 
100 6 0.000565 4 0.000893 19 0.002758 

500 6 0.000904 5 0.001237 20 0.004726 

1000 6 0.001257 a 0.001637 21 0.008809 

5000 6 0.006039 6 0.006157 22 0.056553 

2b (0.2,0.2, +++ ,0.2)7 10 4 0.000594 5 0.002852 15 0.002374 
100 5 0.000736 6 0.000828 15 0.002038 

500 5 0.000884 6 0.001150 16 0.004103 

1000 5 0.001982 6 0.001678 16 0.007631 

5000 5 0.005914 7 0.005991 18 0.045659 


Table 2. Performance profile of M1, M2 and M3 based on iteration number & CPU time (cont) 





Prob ISP Dim M1 M2 M3 
Iter CPU Iter CPU Iter CPU 

3a (0.5,0.5,-+-,0.5)? 10 6 0.000583 4 0.000748 18 0.001662 
100 6 0.000502 4 0.000791 19 0.002683 

500 6 0.000705 5 0.001285 20 0.004508 

1000 6 0.000978 5 0.001404 21 0.007613 

5000 6 0.004791 6 0.005932 22 0.049457 

3b (0.2,0.2, ---,0.2)7 10 4 0.000597 5 0.000625 15 0.001215 
100 5 0.000629 6 0.000764 15 0.001608 

500 5 0.000863 6 0.001065 16 0.003806 

1000 5 0.001085 6 0.002203 16 0.006194 

5000 5 0.005834 7 0.005344 18 0.048274 

4a (0.5,0.5,-+-,0.5)7 10 3 0.000345 1 0.009970 51 0.005631 
100 3 0.000391 1 0.000614 63 0.008490 

500 3 0.000423 1 0.000783 69 0.019573 

1000 4 0.000483 1 0.001370 73 0.032623 

5000 4 0.001014 1 0.004664 719 0.149735 

4b (0.2,0.2, ---,0.2)7 10 2 0.105179 F 0.000493 60 0.006403 
100 4 0.126227 F 0.000784 70 0.013660 

500 4 0.212208 F 0.000963 78 0.022243 

1000 4 0.334854 F 0.005148 80 0.033186 

5000 4 1.369668 F 0.004322 88 0.159905 





The performance profile P: R — [0,1] is defined as follows: Let P and S be the set of problems and 
set of solvers respectively. For n, solvers and n, problems, and for each problem p € P and for each solver 
seéS, ty,s 
is given by: 


:=(number of iterations required to solve problem p by solver s) is defined. The performance ratio 
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Ts t= ty,s/min{ty <}. 


Then the performance profile is defined by: 
de 
P(t) = 7, sizelP E Pin, Sth, 


for all t € R where P(t) is the probability for solver s € S that a performance ratio 7, , is within a factor tT € R 
of the best possible ratio. The code for the proposed method was done using MATLAB 7.1, R2009b 
programming environment and run on a personal computer 2.4GHz, Intel (R) Core (TM) i7-5500U CPU 
processor, 44GB RAM memory and on windows XP operator. Both the methods was implemented with the 


same parameters as y, = 0.01,r = 0.2, o, = o2 = 1074, andyx, = ae The search is stopped if: (i) Il 


F(x,) Il < € with € < 10~* or (ii) The total number of iteration exceeds 1000. 

The meaning of each column in the tables are stated as follows, ’P”: stands for Benchmark problem, 
*ISP”: stands for initial starting points, ’n’’: stands for dimension of the test problems, Iter”: the total number 
of iterations and *CPU”: the processing time in seconds. A particular problem i, performs better if the number 
of iteration (iter) and/or the CPU time in seconds (Time) is less than the number of iteration or the CPU time 
corresponding to the other methods respectively. 

From the above tables, the proposed method outperformed the two other methods in terms of CPU 
times and number of iterations, as shown in Figures | and 2. 
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Figure 2. Performance profile of M1, M2 and M3 with respect to the CPU time 


4. CONCLUSION AND FUTURE WORK 

In this article, a derivative free method for solving symmetric nonlinear systems of equations is given. 
The hybrid method achieved its objectives fully, i.e. it is faster in terms of processing time and effective in 
terms of number of iterations. Thus, the hybrid conjugate gradient parameter is a very good alternative for 
solving symmetric nonlinear systems of equations. For further research work, the method can be modified to 
solve nonsmooth nonlinear equations. 
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